00:00
17:58

Exploring IBM watsonx Prompt Lab

Estimated time: 40 minutes

When it comes to prompting foundation models, there isn't just one right answer. There are usually multiple ways to prompt a foundation model for a successful result.

The IBM watsonx Prompt Lab is a prompting tool designed to help you craft prompts that achieve the desired or expected result from a given model and set of parameters. In this activity, you will explore the key features of the watsonx Prompt Lab, and also learn how to prompt a foundation model.

Accessing IBM watsonx

To access the IBM watsonx platform, you need to have an IBM cloud account. You can use the credentials for the IBM cloud account to access the watsonx platform.

If you do not have an IBM cloud, you will be asked to register and create your account when you try to access the IBM watsonx platform.

Here is the link to the platform: IBM watsonx.

  1. Click the user icon on the upper right corner of the IBM watsonx.ai home screen to log in.
IBM watsonx home page
  1. Log in using the credentials you used while signing up with the feature code.
IBM login page

Note: You might receive a verification code via email to verify and login to your IBM watsonx account.

  1. After logging in, you will land on the home page, where you need to click ‘Start your free trial’ and log in again.
IBM watsonx tour page
  1. Click on Create account or Log in. In the next step, click on Log in, along the text Already have an IBM Cloud account.Choose the same region you had used to create your account.
Create account or Log in Log in with IBM account
  1. Log in again using the credentials you used while signing up with the feature code.
Log in for IBM watsonx

  1. Provide the required information and then click on Continue.

    Log in for IBM watsonx

  2. After the successful login, you will be prompted to Take a tour of the watsonx platform (on your first login), as shown below.

watsonx tour page

Note: You may opt for the watsonx walkthrough by clicking Take a tour or skip or exit it by clicking X at the upper right corner of the screen

  1. On completing or exiting the tour, you will reach your IBM watsonx.ai dashboard page, as shown below. Here, you can perform experiments like train, validate, tune, and deploy AI models.
watsonx dashboard

Note: If this is NOT your first login session in the watsonx.ai, you may find the Prompt Lab deactivated. In this case, you need to create a project and then select the Open Prompt Lab option. You can create a new project by following the given steps.

a. You can view the list of projects for your account using the Projects drop-down option in the left panel.
watsonx projects

b. You can create a new project by clicking the New Project button.
watsonx new project

c. Enter the name and description of the project and click Create.
Create project

d. Go to Home using drop-down option in the left panel.

e. You will reach your IBM watsonx.ai dashboard page.Click the first card with Prompt Lab, as shown in the screenshot below.

IBM watsonx Prompt Lab

f. Click on Associate Service

Associate Service

g. Select Watson Machine Learning-vy and click Associate.If Watson Machine Learning is not seen please select New Service and add Watson Machine Learning

Associate

h. Go to Home using drop-down option in the left panel.

Please be aware that due to specific geographic and business constraints, some may not be eligible to register. If you encounter this limitation, you can still explore the vast landscape of Large Language Models through alternative services such as ChatGPT.

  1. To experiment with a language model, click the first card with Prompt Lab, as shown in the screenshot below.
IBM watsonx Prompt Lab

Note: If you have created a new project, click Open Prompt Lab in the Start working section

  1. You will view the Welcome to the Prompt Lab tour page, as shown below.
Prompt Lab welcome page
  1. You must check all three acknowledgment boxes to start or skip the tour, as shown below.
Welcome to prompt lab tour

Getting Started with watsonx

The Home page should look similar to this screen.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

If you want to work with the Prompt Lab, you will need to work in the context of a project. If you look carefully, you will notice a highlighted square at the bottom of this screen. That's your personal sandbox. In this instance, it's labeled "Vandana's sandbox", as Vandana has signed in to her watsonx account.

Within each project, a suite of tools awaits, enabling you to interact with the AI, construct machine learning models, utilize Jupyter notebooks, and more. Of course, you're welcome to create more projects to organize and structure your work.

If you wish to do that, choose a project from the watsonx.ai home page, and click Open Prompt Lab. This will open the Prompt Lab.

Part A - Exploring the Prompt Lab

Upon entering the Prompt Lab within your project, you'll be greeted by this interface.Select Structured tab.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Familiarize yourself with the tool's features. On the left, you'll find icons representing Sample Prompts, Saved Prompts, and History. By default, these are collapsed.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Additionally, observe the options to create a New Prompt, save your current work, and turn on or off the AI Guardrails. These guardrails are essential safety measures designed to prevent the AI from producing harmful or offensive content. Feel free to explore these features.

Once you've had a good look around, let's delve deeper into the tool's primary components.

At the top, you'll spot three tabs: Chat ,Structured and Freeform. As the name suggests, the Structured tab offers a guided approach to input, reminiscent of a more advanced version of the AI Classroom you've previously used in this course. On the other hand, the Freeform tab presents a simple textbox, allowing you to directly query the AI.

Take time to familiarize yourself with the Structured mode, as it offers a more guided experience.

There are three sections for you in here:

  1. Instruction Section: This is akin to our earlier “Prompt Instructions.”
  2. Example Section: Here, you can provide specific input-output pairs to guide the AI, similar to our exercises in the Chain-of-Thought lab.
  3. Try Section: This is where you'll input your queries. The AI's responses will be displayed in the Output field within this section.

Finally, note the Generate button located at the bottom right. Initially, it appears grayed out, indicating that no input has been provided. Once you enter your query, the button will become active, allowing you to submit your input to the AI for processing.

Exercise 1: Getting Started with Prompt Lab

The following exercises should familiarize you with the Prompt Lab interface. For future convenience, consider saving prompts which you find particularly useful. You can use them as templates, simply tweaking the input field to suit your current query. This method is both efficient and time-saving.Select model flan-ul2-20b

  1. Basic Query: Without any prompt instructions or examples, ask the AI: 
  1. 1
  1. Give me a list of common fish species in North America.

You might find the output lacking. For instance, I received List of fish species in North America. Where's the list? This truncation occurs in part because the default chat setup limits the response to a set word count. Let's rectify this.

  1. Adjusting Model Parameters: Click the Model parameters settings icon in the top right, as depicted below.
Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Adjust the Max token parameter within a range of 1 to 1024. For this exercise, set it to 200, capping our output at 200 words. You can also set a minimum word count and tweak more advanced parameters.

Remember, the cost of using foundation models in IBM WatsonX is determined by usage, which relates to the number of tokens generated. Therefore, setting a conservative Max tokens value is cost-effective.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button. After setting the 200-token limit, close the Model parameters and click Generate again. If the output remains unsatisfactory, don't worry. We have more tricks up our sleeve!

  1. Switching Models: From the dropdown adjacent to Model parameters, switch from the flan-ul2 model to IBM's mixtral-8x7b-instruct-v01-q or granite-13b-chat-v2 model. Click Generate. You should now receive a comprehensive list of common fish species in North America. It doesn't apply here, but it's worth noting that if you're pleased with the output, you can also use the top right corner of the Output field to transfer the output to the Example section. This ensures future responses are modeled after this successful example.

  2. Guardrail Safeguards: You might observe that one entry was omitted due to Guardrail safeguards. Temporarily disable the guardrail at the page's top and generate a new output. Can you identify which panfish species triggered the censor?

  3. Improving English Clarity: Modify the instructions to state: 

  1. 1
  1. Acting as an English professor, you'll take my input, enhance its clarity, and make it sound more native.

Then, input the following fractured English sentence: 

  1. 1
  1. Yesterday, I go to big market for buy many fruits, but they no have the apple I like, so me feel sad and come back home without buy anything.

Experiment with various models. The results should be impressive, making this tool invaluable for non-native English speakers.

  1. Translation: Use the prompt instructions to guide the model to translate your input into Italian or a second language you might speak. Then, input an English sentence. If you do speak a second language, try it with a few different models to see which one performs best. In my case, LLama by Meta did a particularly impressive job with Italian. watsonx is also smart enough to recommend when a specific prompt can benefit from an example, guiding you toward better results.

Part B - Prompting a foundation model

We can now explore in further detail, a few of the functions available on the watsonx Prompt Lab page with a focus on querying foundation models.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Menu List

The menu lists Sample prompts, which you can use. All you need to do is select the task-specific scenario, and the corresponding prompt will appear in Prompt Editor. This is a good way to learn how to prompt a foundation model.

Instructions to use:
Click onsample prompt icon.
Sample prompt may be expanded on the left side of Prompt Editor.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

If you click on Meeting transcript summary, you will see the sample prompts listed for summarizing data from meeting transcripts.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Under Classification, you will find sample prompts to help you sort data based on your preferences.

Model Selection

Another interesting feature is the Model Selection button.
Click on the down arrow to open menu.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

It will list some foundation models and allow you to explore more.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Click on View all foundation models. This will open the Select foundation model dialog box.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Click on any model. This will open a model card or Model Datasheet that contains detailed information about the model.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

To select a model, click on the Select Model button at the bottom right corner of the dialog box.

The model that you have selected will appear in the Model Selection box.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Model and Prompt Parameters

The Model and Prompt Parameters function allows you to specify parameters to control how the model generates output in response to your prompt. Here, you will find a list of parameters you can control in the Prompt Lab.
Decoding
Decoding is the process a model uses to choose the tokens in the generated output. Greedy decoding selects the token with the highest probability at each step of the decoding process. Sampling decoding is more variable and random, which is desirable in creative use cases. However, with greater variability comes the risk of nonsensical output.

Repetition Penalty
If you notice the result generated for your chosen prompt, model, and parameters consistently contains repetitive text, you can try adding a repetition penalty. The higher the penalty, the less likely the result will include repeated text.

Stopping Criteria
You can affect the length of the output generated by the model in two ways: Specifying stop sequences and setting Min tokens and Max tokens. Min tokens must be less than or equal to Max tokens.

Exercise 2: How to prompt a foundation model

Scenario
You work with a cloud software company, and a customer-support chatbot has received a message from a customer. You are tasked with classifying the customer's message as a question or a problem description so the chat can be routed to the correct support team. Let's see how you can adjust the model parameters to achieve this task.

Step 1
Click on the New Prompt button

Step 2
Select a model: either mt0-xxl-13b, flan-t5-xxl-11b, or flan-ul2-20b
The model must only return one of the class names, not be creative and make up new classes.

Step 3
Select the decoding as Greedy.

Step 4
The Repetition penalty is set to 1 by default.

Step 5
Select the Stopping criteria. Specify two stop sequences: "Question" and "Problem". After the model generates either of those words, it should stop.

Step 6
Set the Max tokens parameter to 5.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Step 7
Craft the Prompt text: paste this zero-shot prompt text into the freeform prompt editor in Prompt Lab. 

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  2. *“Classify this customer message into one of two classes: Question, Problem.
  3. Class name: Question*
  5. *Description: The customer is asking a technical question or a how-to question about our products or services.*
  7. *Class name: Problem*
  9. *Description: The customer is describing a problem they are having. They might say they are trying something, but it's not working. They might say they are getting an error or unexpected results.*
  11. *Message: I'm having trouble registering for a new account.*
  13. *Class name: “*
Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Step 8
Click Generate to see the results.

It will be able to classify the message to corresponding Class either Question or Problem.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Exercise 3: Generate a numbered list

Scenario
Say, that you want to generate a numbered list on a given theme using a few-shot prompt.

Step 1
Click on the New Prompt Button.

Step 2
Chose a model: flan-ul2-20b is a good choice.

Step 3
Set the decoding to Sampling and set the following parameters:
1. Temperature: 0.7
2. Top P: 1
3. Top K: 50

Step 4
Set the Random seed to 9045 to get a different output each time you click Generate. Specify a different value for the Random seed parameter or clear the parameter.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Step 5
Identify the Stopping criteria:

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

To make sure the model stops generating text after one list, specify a stop sequence of two newline characters. To do that, click the Stop sequence text box, press the Enter key twice, then click Add sequence. Max tokens parameter to 50.

Step 6
Craft the prompt text: paste this few-shot prompt text into the freeform prompt editor in Prompt Lab. 

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  1. *What are 4 types of dog breed?
  2. 1.Poodle
  3. 2.Dalmatian
  4. 3.Golden retriever
  5. 4.Bulldog*
  7. *What are 3 ways to incorporate exercise into your day?
  8. 1.Go for a walk at lunch
  9. 2.Take the stairs instead of the elevator
  10. 3.Park farther away from your destination*
  12. *What are 4 kinds of vegetable?
  13. 1.Spinach
  14. 2.Carrots
  15. 3.Broccoli
  16. 4.Cauliflower*
  18. *What are the 3 primary colors?
  19. 1.Red
  20. 2.Green
  21. 3.Blue*
  23. *What are 3 ingredients that are good on pizza?”*
Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Step 7
Click Generate to see the result.

Select Groups window where you search groups, highlightes Administrators group and the Check Names button.

Three ingredients are suggested.

Conclusion

The Prompt Lab is a versatile tool. Its AI models have helpful capabilities such as generating creative content, answering complex queries, assisting in language translation, summarizing content, extracting specific data from unstructured data, and even simulating specific roles like an English professor, as seen in this lab. It's a testament to the power and utility of modern AI.